Perturbed Proximal Primal Dual Algorithm for Nonconvex Nonsmooth Optimization

In this paper we propose a perturbed proximal primal dual algorithm (PProx-PDA) for an important class of optimization problems whose objective is the sum of smooth (possibly nonconvex) and convex (possibly nonsmooth) functions subject to a linear equality constraint. This family of problems has applications in a number of statistical and engineering applications, for example in high-dimensional subspace estimation, and distributed signal processing and learning over networks. The proposed method is of Uzawa type, in which a primal gradient descent step is performed followed by an (approximate) dual gradient ascent step. One distinctive feature of the proposed algorithm is that the primal and dual steps are both perturbed appropriately using past iterates so that a number of asymptotic convergence and rate of convergence results (to first-order stationary solutions) can be obtained. Finally, we conduct extensive numerical experiments to validate the effectiveness of the proposed algorithm.